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This study from the city school district in Saginaw, 
Michigan measures the effects of the Chapter 1 Academic Achievement 
program on fifth grade students during the 1981-82 school year. A 
total of 463 students were classified into four categories depending 
on whether they were single yesr, multiple year, or continuous 
participants in the program; or whether they were from the regular 
education program. Three different standards were used to gauge the 
growth of the three groups receiving compensatory education: (1) 
normal curve equivalent; (2) normal growth; and (3) relative growth. 
The single and multiple year groups failed to meet the criterion 
score in reading, while the continuous group exceeded it. All three 
groups exceeded normally expected reading growth but failed to do so 
in mathematics. On the index that compared the compensatory groups to 
the regular group, only the continuous group showed a decrease in the 
gap between their group and the comparison group. The recommendations 
were the following: (1) future studies such as this one should be 
outlined in advance to assure that accurate longitudinal records are 
kept; (2) special studies are needed for students who test high but 
are still in need of compensatory education; and (3) these kind of 
data should be collected state-wide and shared with all of the 
educational community. Tables, appendices providing calculation 
instructions in gap reduction research design, and a brief 
bibliography are included. (VM) 
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This study was designed to achieve two primary goals. The first goal was 
to evaluate the long-tern sustained Ispact of the Chapter 1 Compensatory Edu- 
cation Program operated by the School District of the City of Saglnav on both 
former particlpatlf^g pupils and pupils who continued to participate because of 
low achleveaent. A second goal was to meet the evaluation requirement of 
Chapter 1 of the Education Consolidation and Improvement Act (ECIA) of 1981 
which mandates that once every three years (after the 1978*79 school year) a 
determination be made as to whether Improved performance Is sustained over a 
period of more than one year from programs funded under this act.^ The Long- 
Term Sustained Effects of Chapter ^ Participation, 1979-1982 was the first 
report which focused on satisfying these two goals. This first report tracked 
the performance of compensatory education students from third grade through 
fifth grade. 

What follows Is the second report In the series that starts with fifth 
grade students who have participated In the compensatory education program at 
least one of the prior three y^ars (as third, fourth, or fifth graders) and 
then tracks their performance through the eighth grade. However, before 
covering the precise details of the study an overview of Saglnaw^s Chapter 1 
Compensatory Education Program Is necessary to put the details of the study 
In the proper perspective. 



The funding legislation was originally known as Title I of the Elemen 
tary and Secondary Education Act of 1965. 



The Saginaw prograa funded under Chapter 1 is currently entitled Aca- 
2 

dealc Achleveaent (A )• The puLpose of this prograa la to Improve the read- 
ing and aatheaatlcs achleveaent of a designated number of educationally dls* 

2 

advantaged students. The Chapter 1 funded A prograa ferved approximately 

2,00(H> students In grades R*9 each of the six years since 1979^. 

Both product and process evaluations of the prograa have been conducted 

for the past six years. The product evaluation reports describe the academic 
2 

achleveaent of A students fully md are available upon request froa the 

Department of Evaluation^ Testing and Research. Grade level achleveaent of 
2 

A students according to the performance standard are summarized for both 
subject areas In the chart belov for the six school years of Interest. 



The 1985** {6 school year was the first year the compensatory education 
prograa was evaluated under the naae of Academic Achievement (A ). Our prior 
evaluation reports have referred to the same prograa as the Supplemental 
Teacher Participation (STP). 

3 

Eighth grade students were served since the 1981-82 school year and 
ninth grade students have been served since the 1982-83 school year. 
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Overall lecord of Chapter 1 Prograa AttaiiHieiits from 1979 to 1985 

AttaioMot of Pterfonaiice Standard^ 



Grade Subject 

K Reading 

Hatheaatlcs 

1 Reading 
Hatheaatlcs 

2 Reading 
Hatheaatlcs 

3 Reading 
Hatheaatlcs 

4 Reading 
Hatheaatlcs 

5 Reading 
Hatheaatlcs 

6 Reading 
Hatheaatlcs 

7 Reading 
Hatheaatlcs 

8 Reading 
Hatheaatlcs 

9 Reading 
Hatheaatlcs 

Suaaary of Perforaance 
Standard Attalnaent 



1979-1980 


1980-1981 


1981-1982 


1982-1983 


1983-1984 


1984-1 


Yes 


No 


Yes 


Yes 


— 


— 


Yes 


No 


Yes 


No 


— 


— 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 




Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


No 


No 


Yes 


No 


Yes 


No 


No 


No 


No 


No 






Yes 


Yes 


Yes 


No 






Yes 


Yes 


Yes 


Yeb 








Yes 


Yes 


Yes 








Yes 


Yes 


Yes 


Yes (lOO.OZ) 


9-Yes (56. 2Z) 


16-Yes (88. 9Z) 


17-Yes (85. OZ) 


16-Yes (88.9Z) 


12- Yes (66. 


No ( O.OZ) 


7-No (43. 8Z) 


2-No (11. IZ) 


3-No (15.0Z) 


2-No (11. IZ) 


6-No (33.. 



1 



Attalnaent of perforaance standard Is defined as an laproveaent of the aean post-test percentile score over 
aean pre-test percentile score. 
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Overall, the achievement levels In terms of the attainment of the per-^ 
formance standard for the A program have been very positive . Achievement of 
the performance standard was the best during 1979-80 with lOOZ positive 
attainment followed by 1981-82 and 1983-84 with 88,9J: positive attainment, 
1982-83 with 85.01 positive attainment, 1984-85 with 66.71 positive attain- 
ment, and 1980-81 with 56.22 positive attainment. Student achievement levels 
for reading and mathematics seemed quite similar In terms of attaining the 
performance standard over the six year period. One definite pattern, despite 
overall success. Is the lack of achievements of the program at the seventh 
grade level. 

It Is In this context that the Department of Evaluation, Testing, and 

Research has attempted to Implement an Investigation Into the sustained effec- 

2 

tlveness of Chapter 1 A program on participants who obtained differential 

2 

lengths of service from the A program. 



Hullln and Summers (1983) studied all the ''major'* compensatory education 
studies through 1982. Generally their review Indicated that compenpatory edu- 
cation programs have a positive though small effect on the achievement of dis- 
advantaged students. Our findings locally through our first sustained effects 
study showed much larger positive gains (at or above "normal growth") ^cross 
the majority of grade levels studied. 
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The cohort of Chapter 1 pupils who had been In the A program as third 
graders during 1979-80 school year, fourth graders during 1980-81 school year, 
and/or fifth graders during the 1981-82 school year were chosen as the sub- 
jects for the study. The choice was made for the following reasons: 1) many 
of these same pupils were In the original sustained effects study, 2) test 
data on the majority of the pupils would exist at the end of the three year 
period because dlstrlct-vlde testing occurs both In the spring of fifth and 
eighth grades, and 3) the junior high years typically are where accumulative 
academic deficits are the most pronounced In Chapter 1 programs. Academic 
achievement gains even for the regular K-12 education population are among the 
smallest during this period. The decision to save data for this study was 
made during the 1982-83 school year but the exact nature of the study was not 
finalized until the Summer of 1987. 

The problem addressed by the study was to determine If the A program 
made a long-term sus) alned Impact In terms of reading and mathematics achieve- 
ment as measured hy the 1977 version of the California Achievement Tests — Form 
C (CAT). Three different standards were used as criteria to determine the 
presence or absence of a long-term sustained effect. All three criteria (NCE 
gains, normal growth, and gap reduction) deal with the concept of gain fc^n: a 
pre-test to a post-test measurement of reading and mathematics achievement. 

The first criterion chosen was a normal curve equivalent (NCE) score gain 

equal to or greater than the gain of fifth grade pupils who were In the A 

program from the Spring 1982 through Spring 1985 as eighth graders. Partlcl- 
2 

pants In the A program had to have pre-test scores at or below the 44 NCE to 
reoLaln eligible for one or more of the three prior years (1979-80, 1980-81, 
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and 1981-82). The sustaining effect pupils were pre-tested (Spring, 1982) and 
finally post-tested (Spring, 1985) on CAT. 

The second criterion was **noniial growth** (Echtemacht, 1980). Normal 
growfJ defined as an estimate of how well students will perform In the 
ai. ^ of any special program. Spring, 1982 CAT results served as the pre- 
test and Spring, 1985 CAT test results served as the post-test wh^n the con- 
cept of "^normal growth** was employed to determine the percent of gain beyond 
**normal growth**. (See Appendix A for the calculation of normal growth In 
reading and mathematics.) The additional growth In comparison to **normal 
growth** was another standard chosen as a measure that growth was sustained. 

The final criterion Involved the calculation of a Relative Growth Index 
(R6I). This Index Indicates the percentage Increase or decrease of the sus- 
taining effect group (treatment group) and a regular education group with no 
prior compensatory education participation (comparison group) between the mean 
pre- and post-test achievement levels. It Is expected (or at least hoped) 

that the gap will stay the same (sustaining) or be reduced as a result of 
2 

prior A program participation and thus will be the same or smaller at post- 
test time than It was at pre-test time. Figure 1 below Illustrates the 
reduced gap expected between treatment (T) and comparison (C) groups. 
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Pre-Test 



Post-Test 
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To calculate the Relative Growth Index (RGI), the comparison group^s pre- 
and post-teat standard deviations are pooled. This pooled standard deviation 
Is the metric In which growth estimates for the project and comparison groups 
are cast. Finally , the growth of the project group Is expressed as a per- 
centage of the growth of the comparison group, thus providing an easy-to- 
Interpret Relative Growth Index (RGI). (See Appendix B for the steps Involved 

In the calculation of this Index.) RGI's less than 100% Indicate th$t the 
2 

former A students fell further behind the non-partlclpants during the three 

year study period. RGI^s equal to lOOZ Indicate that the project group grew 

at the same rate as regular education students, and RGI^^s greater than lOOZ 

2 

Indicate that former A project participants out galucd the non-partlclpants. 

2 

The results of two different lengths of A participation ( multiple year 

and single year participation ) and two different comparison groups ( regular 

2 

education students and continuous A participants ) were the primary focus of 
the study. The chart below Illustrates the number of students In each group 
and their participation or lack of participation In the Chapter 1 program from 
1979 to 1985. 
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lm^T4 of BirtielpatloQ of Varloas Groups Inyolved in the 
SMtained Effects Stody 

School Tear 



Group Naae 


Nuaber of 
Students 


1979-80 


1980-81 


1981-82 


1982-83 1983-84 1984-85 


Multiple Year I /stained 
Effect Students 


1 
5 
16 
2 


X* 

X 
X 


X 
X 
X 


X 
X 

X 


SUSTAINED PERIOD 


Single Year Sustained 
Effect Students 


4 
44 

31 


X 


X 


y 




Regular Education Com- 
parison Students 


317 










Continuous Coapensatory 
Education Students 


43 


X 


X 


X 


XXX 



*X « Tear of participation In Chapter !• 



As can be seen In the chart above the multiple year sustaining effect, 
single year sustaining effect, regular education comparisons, and continuous 
compensatory education student groups consisted of 24, 79, 317, and 43 stu- 
dents respectively. It had been contemplated that the study deal with all the 
possible lengths of differential participation charted above, however, because 
of the lack of sufficient numbers In the seven groups these were collapsed 
Into the multiple and single year of participation groups. 

These two levels of participation (multiple or single) are fundamental to 
the statement of the study hypotheses. In addition, the effect of continuous 
participation for six years will be explored through the stated hypotheses. 
The study hypotheses stated below are grouped according to the criterion cate- 
gories discussed earlier. 
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HTPOIHESBS: HCB 



1. There vlll be NCE growth equal to or exceeding the regular 
education coaparlson group students In reading and mathe- 
matics as measured by CAT for the multiple year partlclpa*- 
tlon group. 

2. There vlll be NCE growth equal to or exceeding the regular 
educational comparison group students In reading and mathe- 
matics as measured Hy CAT for the single year participa- 
tion group. 

3. There i^lll be NCE growth equal to or exceeding the contin- 
uous A participant group students In reading and mathe- 
matics as measured by CAT for the multiple year participant 
group. 

4. There ^111 be NCE growth equal to or exceeding the contin- 
uous A participant group students In reading and mathe- 
matics as measured by CAT for the single year participant 
group. 

5. There will be NCE growth equal to or exceeding the regular 
education comparison group students In readlng2and mathe- 
matics as measured by CAT for the continuous A students. 



BTPOngSES; HORMAL GROWTH 

1. There will be growth equal to or exceeding **normal growth" 
In reading and mathematics as measured by CAT for multiple 
year participants. 

2. There will be growth equal to or exceeding **normal growth" 
In reading and mathematics as measured by CAT for single 
year participants. 



2 Multiple years participation were those pupils with two or three years 
of A participation from Fall, 1979 to Spring, 1982. They could have had 
additional years of participation prior to Fall, 1979. 

^Single year participants were those pupils with a single year of par- 
ticipation from Fall, 1979 to Spring, 1982. They could have had additional 
years of participation prior to Fall, 1979. 

7 2 

Continuous A^ participant group students were those pupils having par- 
ticipated In^the A program from Fall, 1979 to Spring, 1985 with six years of 
continuous A participation. Again, they could have had additional years of 
participation prior to Fall, 1979. 
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BTPOIHESES: UIATIVE GBORH INDEX 



1, There will be « Relative Growth ladex (R6I) of 100% or 
greater In reading and aatheiiiatlcs as measured by CAT for 
nultlple year participants. 

2, There will be a Relative Growth Index (RGI) of lOOZ or 
greater In reading and aatheaatlcs as measured by CAT for 
single year participants. 

3, There will be a Relative Growth Index (RGI) of lOOZ or 
greater In reading and mathenatlcs as measured by CAT for 
continuous compensatory education comparison group. 
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What follotfs l8 a presentation of the CAT reading and mathematics results 

2 

contrasted vlth the two different levels of A participation according to the 
three grovth standards. The first criterion used to contrast growth levels Is 
NCE gains In terms of regular and continuous compensatory education comparison 
groups. The next criterion used to contrast growth levels Is "normal growth". 
The third criterion was the Relative Growth Index (RGI). 

■CE Gmlm Criterion Immultm 

As NCE gains are used by the State and Federal government as the primary 
way to report annual Chapter 1 results, It would seem that such a mea.iurement 
of academic growth Is also viable for this study. Tables : and 2 below pre- 
sent another way to denote achievement gains by the compensatory education 
groups, using NCE gains by group for both rjeadlng and mathematics. Table 1 
below presents the reading gains over the three year period of study. 



TABLE 1. GAU/LOSS IH milAL CII1?E EQDIVALER OMITS— tEADIK. 







Normal Curve Equivalents 


Group 


Number 
Tested 


Pre-Test Mean 
Spring, 1982 


Post-Test Mean 
Spring, 1985 


Gain/ 
Loss 


Multiple Year Sustaining 
Effect Students 


24 


53.67 


47.04 


- 6.63 


Single Tear Sustaining 
Effect Students 


79 


55.18 


50.87 


- 4.31 


Regular Education Compari- 
son Students 


317 


66.39 


62.73 


- 3.66 


Continuous Compensatory 
Education Students 


43 


38.44 


38.47 


+ 0.03 
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An exaoilnatloa of Table 1 reveals that three groups — multiple year, 
single year, and regular education students experienced losses in reading 
of -6.63, -4.31, and -3.66 noraal curve equivalent units. Only the continu* 
ous coapensatory education student group experienced a gain of 0.03 normal 
curve equivalent units. Thus both single and multiple sustaining effect 
groups failed to match the NCE performance of either the regular or continu- 
ous compensatory education student groups. The continuous compensatory edu- 
cation group did out perform the regular education comparison groups in read- 
ing. 

The mathematics results are presented in Table 2 below. The total net 
gain in NCE'^s of regular and continuous compensatory education groups again 
will serve as the criteria for comparison. 



TABLE 2. Gka/lOSS H MUIAL CORfE EQVIVALER OUTS— MATHEMATICS. 







Honul 


Curve Equivalents 


Group 


Nuaber 
Tested 


Pre-Test Mean 
Spring, 1982 


Post-Test Mean 
Spring, 1985 


Gain/ 
Loss 


Multiple Year Sustaining 
Effect Students 


24 


43.54 


38.08 


- 5.46 


Single T^ar Sustaining 
Effect Students 


79 


50.57 


44.97 


- 5.60 


Regular Education Coaparl- 
son Students 


317 


62.14 


58.59 


- 3.55 


Continuous Coapensatory 
Education Students 


43 


32.28 


28.98 


- 3.30 



A review of matheaatlcs results reveals that all four groups— Multiple 
year, single year, regular, and continuous education students — showed losses 
of -5.46, -5.60, -3.55, and -3.30 respectively. Again, both multiple and 
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single sustalnlog effects groups failed to match the NCE performance of either 
the regular or continuous compensatory education student groups. The contln* 
uous compensatory education student group performance (-3.30) did exceed the 
performance of the regular education group (-3.55) In mathematics. 

The follovlng chart summarizes which sustaining effect and continuous 
compensatory education groups grew at a rate equal to or exceeding the com- 
parison group of Interest In reading and mathematics. The chart also relates 
these questions back to the hypotheses stated earlier. 



Hypothesis Results Which Equal or 

^ Hypothesis Exceed Gains Hypothesized 

Read Ing/Na t hema t Ics 

1 Multiple > Regular Education No nq 

2 Single > Regular Education No No 

3 Multiple > Continuous No No 

4 Single > Continuous No No 

5 Continuous > Regular Education Yes Yes 



As Indicated above both sustaining effect groups failed to equal or 

exceed the NCE gain/loss of the comparison groups In either reading or mathe- 

2 

matlcs. The continuous A compensatory education student group did exceed the 
growth of regular education students In mathematics and reading. 

. 

"lormml Growth* Criterion ■camlts 

Table 3 below compares Chapter 1 results In reading and math with normal 
growth for compensatory education students by category. The term "normal 
growth- Is an Index which standardizes the scores for an entire grade level, 
thereby allowing comparisons with any subgroup of Interest (see Appendix A for 

14 
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the calculation of normal growth). These data Incorporate a time span of 
Spring, 1982 to Spring, 1985. 



TABLE 3. OOMPAUSOH OF GAlBOOtT 6B0UP OtOHTH TO KUIAL GROillH. 



Subject Area/ 
Growth Category 


Compensatory Education Category 


Multiple 
(N-24) 


Single 
(N-79) 


Continous 
(N-43) 


Reading 








Normal Growth 
Category Group Growth 
Normal - Category Group Growth 
Z Additional Growth 


22.1 
23.1 
1.0 
4.5Z 


22.1 
26.8 
4.7 
21. 3Z 


22.1 
24.9 
2.8 
12. 7Z 


Mathematics 








Normal Growth 
Category Group Growth 
Normal - Category Group Girowth 
Z Additional Growth 


25.3 
21.4 
- 3.9 
-15.4Z 


25.3 
22.3 
- 3.0 
-11. 8Z 


25.3 
21.3 
- 4.0 
-15. 8Z 



As Indicated In Table 3, noraal growth for coaipensatory education stu- 
dents from fifth to eighth grade In reading attained a value of 22.1 NCE's. 
Using this nuaber as our baseline, both multiple and single year sustaining 
effect pupils shoved growth above normal growth of 1.0 NCE's (or 4.5Z) and 4.7 
NCE's (or 21. 3Z) respectively. The continuous compensatory education group 
showed growth above normal growth of 2.8 NCE's (or 12. 7Z). 

Table 3 also presents the same Information for mathematics . Again, nor- 
mal growth Is the standard for comparison. Normal growth had a value of 25.3 
MCE's over the three year span. The multiple year sustaining effect students 
achieved a growth of 21.4 NCE's or 15.4Z less than normal growth. The single 
year sustaining effect group showed a growth of 22.3 NCE's or 11. 8Z less than 

15 
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Qoraal grovth. The continuous coapensatory education group attained a growth 
of 21.3 NCE^s or 15.8Z less than normal growth. 

The following chart designates which sustaining effect and continuous 
coapensatory education groups grew at a rate equal to or exceeding normal 
growth In reading and matheaatlcs. The chart also relates these quesw^ons 
back to the hypotheses stated earlier. 



Hypothesis Results Which Equal or 

* Hypothesis Exceed Gains Hypothesized 

Re ad 1 ng/Ha t hema 1 1 cs 

1 Multiple > Normal Growth Yes No 

2 Single > Normal Growth Tes No 

3 Continuous > Normal Growth Tes No 



As Indicated above both sustaining effect groups and the continuous 
compensatory education groups equalled or exceeded normal growth In reading 
while all three groups failed to equal normal growth In mathematics. 

telatlve Grwrth Index (ICI) Criterion tcamlta 

The final criterion for comparison purposes was the Relative Growth Index 
(R6I). The RGI Is the statistic used In the gap reduction evaluation model 
design. The research question posed Is "Whether the project group (sustain- 
ing effect students) Is catching up to, keeping up with, or falling behind the 
comparison group (students having no Chapter 1 services for a six year 
period) The gap measured Is the gap between the mean achievement level of 
the sustaining effect group and the mean achievement level of the regular edu- 
cation comparison group. It Is hypothesized that the gap between the sustain- 
ing effect and regular education comparison groups will remain the same or be 
reduced between pre- and post-testing. To evaluate this hypothesis the 
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regular education group^'s pre- and post-test standard deviations are pooled. 
This pooled standard deviation Is then used as the metric In which growth 
estlaates for the project and conparlsoc group are liieasured. Finally, the 
growth of the project group Is expressed as a percentage of the growth of the 
conparlson group, thus providing an easy-to-lnterpret RGI (see Appendix B for 
the exact steps to calculate the Relative Growth Index). 

The Interpretation of the RGI deserves a bit of an explanation. A RGI 
less than lOOZ Indicates the project group (or In our case the sustaining 
effect group) Is falling behind the comparison group. When the RGI equals 
lOOZ It signifies the project group Is keeping equal to the regular education 
comparison group. A RGI greater than lOOZ means the project group Is catching 
tq> to the regular education comparison group. Figure 2 puts this Interpreta- 
tion In graphic form relative to the gap between the project and comparison 
group. 



• KGI Icsi Chan lOOZ 

signifies falling behind 
coaparlson group 



High 



Test 
Score 



Low 




cofflperlson group 
project group 



• RGI €quel Co lOOZ 
signifies keeping up 
vlch comparison group 



High 



Test 

Score 



Low 



comparison group 
project group 



-J U- 
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• ^^'^ grester than lOOZ 
signifies catching up 
to conparlson group 



High 



Test 
Score 



Low 



coapsrlsoti group ' 
project group 
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Tabid 4 below presents the RGIs for the multiple and single year sustain- 
ing effect groups plus the continuous compensatory education students for both 
reading and mathematics* 



TABLE 4« BELmVB OOVIH HDICBS (IGIa) BT CBOUP Dl 
lEADIMC AID HATHEHATICS. 



Group 


Relative Growth Indices 


Reading 


Mathematics 


Multiple Tear Sustaining 
Effect Students 


86.85Z 


93.60Z 


Single Tear Sustaining 
Effect Students 


95.20Z 


90.25Z 


Continuous Compensatory 
Education Students 


131. 82Z 


102. 70Z 



A review of Table 4 reveals that both the multiple and single year sus- 
taining effect students failed to equal or exceed the growth of the regular 
education students. Sustaining effect single year students did the best in 
reading (RGI - 95.20Z) and then in mathematics (RGI - 90.25Z). Multiple year 
sustaining effects students did the best in mathematics (RGI « 93.60Z) and 
then reading (RGI « 90.25Z). The continuous compensatory education group 
exceeded the regular education comparison group in both reading (RGI « 
131.82Z) and mathematics (RGI « 102. 70Z). 

The following chart specifies the hypotheses relating to the RGI's and 
their status in the areas of reading and mathematics. 
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Hypothesis 
# 



1 
2 
3 



Hypothesis 

Multiple > lOOZ RGI 
Single > lOOZ RGI 
Continuous > lOOZ RGI 



Results Which Equal or 
Exceed Gains Hypothesized 

Reading/Mathematics 

No No 

No No 

Tes Yes 



As Indicated above both sustaining effect groups failed to equal or 

exceed the specified RGI of the regular education coaparlson group In either 

2 

reading or mathenatlcs. The continuous A participants exceeded the lOOZ RGI 
In both reading and mathematics. 
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DISODSSIOM 

The datA presented on the preceding pages provided three standards used 
for comparison of the continuous and sustained effects of Chapter 1 partlclpa* 
tlon. What follows Is an Interpretation of the analyses presented earlier. 

Chapter 1 students have demonstrated an academic deficit In reading 
and/or mathematics. Participation In the program Is provided to students who 
score at or below the 44 NCE on. CAT. Since standardized testing provides the 
district with a means to Identify potential students, the use of test results 
should be considered as a source of Information to document long-term effec- 
tiveness. As eligibility Is reassessed each year, those who remain eligible 
continue to exhibit the greatest need. Consequently, continuous (those stu- 
dents who remain In the Chapter 1 program the longest), multiple year, and 
then single year participants do show a constant lower level of absolute 
achievement In comparison to each other and regular education students. The 
expectation might then be, that these students would also show smaller Incre- 
ments of growth. However, It can be seen (see Tables 1 and 2) that Chapter 1 

continuous participants show the greatest percentage of growth® In comparison 
2 

to the other A groups studied the longer they remain In the program without 
Interruption. The presumption that academically disadvantaged children who 
continue to be placed In compensatory education programs grow at lower rates 
than normal may be erroneous according to the data In Table 4. Succinctly 



g 

The authors acknowledge that a small amount of net gain (0.5 MCE or 
less) reported by the various groups may be aue to the regression effect 
(Roberts; 1980, pp. 78-80). However, the magnitude of the error Is relatively 
small so that resultant gains still reflect a positive effect of A program 
participants. Urn (1980) and Burton (1980) address the regression effect 
topic In greater a«tall for the Interested reader. 
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stated, our present expectation Is that Chapter 1 participants experience 
greater gains the longer they continuously participate In the A program. 

The Issue of sustained effects after Chapter 1 participation stops was 
the other question examined. It appears from 2 of the 3 criteria used that 
growth Is not sustained by either former single or multiple year A partici- 
pants. However, those students who do continue to participate In Chapter 1 
tend to show the greatest percentage of growth In comparison to other groups. 

This study represents the second effort by the Saginaw Public Schools to 
look at sustained effects of both ChapteAT 1 participants and former partici- 
pants. Based on this second experience, It Is apparent that a number of Inde- 
pendent variables need to be controlled In subsequent studies. These vari- 
ables Include the following: variations among Chapter 1 program sites; his- 
tory of students; participation In other compensatory education programs; 
better Identification of participants; summer vacation effects; etc. Hope- 
fully, this school district and others, with help from the Michigan State 
Department of Education will do other studies to more definitively determine 
the long-term continuous and sustained effects of Chapter 1 participation. 

It was found that the when compensatory education services are stopped 
(or Interrupted ), these students lose academically In both subject areaa 
either due to not having a chance to further consolidate learnings or because 
the measurement of their achievement showed Incorrectly high levels of 
attainment. The sustaining effect group (and the Interrupted group), however, 
showed a smaller drop In reading than In mathematics possible due to the fact 
that reading skills are more likely to be dealt with on a regular basis 



The Interrupted service Issue was explored In Saglnaw^s first study of 
Long-Term Sustained Effects of Chapter 1 Participation, 1983-85 . 
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outside the school envlronaent. Consequently^ Incidental learning of reading 
related skills Is more likely to take place. Thus, the best achievement 
results cotse froa Chapter 1 students with continuous rather than Interrupted 
participation . This Is a conclusion consistent with the findings of the pre- 
vious Chapter 1 sustained effects study, 

A great deal of Information has been shared with the reader thus far. At 
this point an effort will be made to summarize this Information and formulate 
concluding statements. 
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SDIHAET AID OOKLOSIOIS 



A study of the loag-ten continuous and sustained Impact of the Chapter 1 
Academic Achievement (A ) program on former third to fifth grade participants 
during the 1979-80 through 1981-82 school years vas undertaken. A total of 
463 students were tracked over the six year period of the study. One hundred 
forty students uere In the treatment groups studied (single year « 79, multi- 
ple year « 24, and continuous " 43). The remaining 317 students were In the 
regular education comparison group. The participation categories were the 
following: single year, multiple year (2 or 3 years), and continuous (6 
years). 

Three different standards were used to gauge observed growth of groups In 

each category. The first criterion chosen was a normal curve equivalent (NCE) 

score gain equal to or greater than the gain of fifth grade pupils who were In 
2 

the A program from the Spring, 1982 through Spring, 1985 as eighth graders. 
The single and multiple groups failed the NCE gain criterion while the con- 
tinuous compensatory student group exceeded the criterion In reading. 

The second criterion was "normal growth"* (Echtemacht, 1980). Normal 
growth Is defined as an estimate of how well compensatory education students 
will perform In the absence of any special program. All three groups (single, 
multiple, and continuous) out-performed normal growth In reading but failed to 
do so In mathematics. 

The final criterion Involved the calculation of a Relative Growth Index 
(HGI). This Index Indicates the percentage Increase or decrease of the treat- 
ment (study groups) and the regular education group with no prior compensatory 
education participation (comparison group) between the mean pre- and post-test 
achievement levels. Only the continuous compensatory education group 
decreased the gap between themselves and the regular education students 

23 



between pre- and post-testing. The decreased gap (or the RGI In excess of 

lOOZ) for the continuous group occurred In both reading and mathematics 

(131. 82Z and 102. 70Z respectively). 

Overall, It was found that the academic achievement of Saglnaw^s Chapter 

1 population Is associated positively with the amount of continuous time spent 
2 

In the A program . 
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IBOaGHEHDAIIONS 



Llstad below are a series of recoamendatlons based on the findings 
this study. These recoaaendatlons are offered In an effort to Improve 
long-ten lapleaentatlon and lapact of the Chapter 1 program. 



1. Definite plans for any future long-ten continuous 
and sustained effects studies should be outlined In 
advance to ensure that accurate longitudinal records 
of Chapter 1 participants are available. Such records 
would yield more comprehensive accounting of all 
aspects of participation (past history of participa- 
tion, longitudinal test file linkage with unique stu- 
dent numbers for each student, detenlnatlon of 
research/evaluation questions of Interest, necessary 
testing points to answer questions posed, etc.) and 
ultimately make possible better understanding of the 
nature of any sustained effects through better con- 
trols. 

2 

2. Special study of higher scoring A participants who 
test-out but still seem In need of compensatory edu- 
cation services In subsequent years should be planned 
for and undertaken, iguch a study would focus on the 
following questions. 

e Are higher achievers (relatively speaking) showing 
high scores because of a large error component to 
their test scores due to some testing related fac- 
tor (e.g., guessing. Improper test administration, 
etc.)? 

e Do higher achieving students possess any readily 
observable characteristics that set them apart as 
a group? 

# Do these students appear more frequently In cer- 
tain buildings than In others? 

e How much and what type of additional help should 
this group of pupils be given (If they can be 
Identified) to foster lasting achievements? 



This recommendation which comes from Sagloaw^s first Long-Ten Su 
talned Effects Study and still seems relevant as of this writing. 
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• Can an alternate cut-off criterion score oe eatab 
liahed to ensure fever false positives? 

3. The Hichigan Departsent of Education should collect 
the existing long-tens continuous and sustained 
effects studies state*-ffide and sunaarised results 
should be shared vith the rest of the Staters edu- 
cators. Also, suggestions on relevant issues and 
possible Methodologies to use in such studies would 
be helpful to both saall, aediun, and large school 
districts. 
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APPENDIX A 



CALCDLmOH OF *I0I1IAL GKOHIH" 



STEP 1: Detenlne average percentile score 
of entering fifth grade pupils. 

STEP 2: Look up the scale score that cor- 
responds to this percentile In the 
spring of the jear at both the 
eighth and fifth grade levels. 



STEP 3: Subtract the fifth grade scale 

score froa the eighth grade scale 
score to obtain the aaount of gain 
In scale score units. 

STEP 4: Divide this gain bjr the standard 

deviation of the eighth grade group 
In scale score units to obtain the 
pre- to post*test gap. 

STEP 5: Multiply this gap by the standard 
deviation of the NCE distribution. 
I.e. 9 standard score system vlth 
standard deviation set equal to 
21.06, to obtain the estimate of 
the amount of ^normal growth** In 
NCE units between fifth and eighth 
grade for compensatory education 
students. 



Reading > 
l6Zlle 

Reading 
5th grade 
leZlxe - 406 
8th grade 
16Zlle - 476 

476-406 - 70 



1.05 X 21.06 
- 22.11 



Hatheaatlcs * 
23Zlle 

Hatheaatlcs 

5th grade 
23Zlle - 429 
8th grade 
23Zlle - 500 



500-429 - 71 



71 
59.1 



- 1.20 



1.20 X 21.06 
- 25.27 
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APPENDIX B 



STEPS TO CALCDLiTE THE ULATIVE GBOVTH UDEZ Dl THE GAP 
BEDDCnOH EBSIAEGH DBSId 

STEP 1: (For use with a standardised achlevenent test.) Convert each 

project and comparison group student's raw pre-test and post-test 
scores to scale scores using the correct conversion table for the 
focB and level of the test used. If a non-standardized test was 
i^sed skip this step. 

STEP 2: Coapute the aean pre-test and post-test (rav or. If the test has 
thea, scale) scores of project students at each grade level. 

STEP 3: Coapute the aean pre-test and post-test scores of comparison group 
students at each grade level. 

STEP 4: Compute the pre-test and post-test standard deviations of comparison 
group students at each grade level. 

STEP 5: Subtract the project group'^s mean pre-test score from the comparison 
group^s mean pre-test score. Divide the difference by the compari- 
son group^'s pre-test standard deviation and label the result the 
pre-test gap. 

STEP 6: Subtract the project group^s aean post-test score from the compari- 
son group^'s mean post-test score. Divide the difference by the com- 
parison group^s post-test standard deviation and label the result 
the post-test gap. 

STEP 7: Subtract the post-test gap (from Step 6) from the pre-test gap (from 
Step 5) and label the difference the gap reduction. (The gap reduc- 
tion may be negative. Be sure to keep track of the sign!) 

STEP 8: Subtract the comparison group^'s mean pre-test score from Its mean 
post-test score and label the difference the comparison group'^s 
unstandardlzed growth estimate. 

STEP 9: Oslng thn comparison group'^s pre- and post-test standard deviations, 
calculate the following value: 



V 



(S.D. )^ + (S.D. )^ 



post 



2 

Label this value the coapirlsoa group's pooled standard deviation. 
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STEP 10: Divide the coaparlson group^s un standardized growth estimate (from 
Step 8) by the coaparlson group's pooled standard deviation (from 
Step 9). Label this value the comparison group's standardized 
growth estimate. 

STEP 11: Add the gap reduction (from Step 7) to the comparison group's stan- 
dardized growth estimate (from St^.p 10). Label this sum the project 
group's standardized growth estimate. 

STEP 12: Divide the project group's standardized growth estimate (from Step 
11) by the comparison group's standardized growth estimate (from 
Step 10). Multiply the result by 100 to convert It to a percent and 
label It the Relative Growth Index (RGI). 



\ 
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APPENDIX B 



CALCOLATIOH OF THE RELATIVE GKOHTH IHDEX (BGI) IH THE GAP 
lEDDCTIOl IBSBAftCH DESIGN 



An example nay help operatlonallze the steps on the preceding two pages. Con- 
aider the following data set for a particular grade level. 



STEP 5: 
STEP 6: 
STEP 7: 
STEP 8: 

STEP 9: 



Pretest Mean 
Pretest Std. Dev. 
Post test Mean 
Posttest Std. Dev. 



Project 
Group 

355.34 

N/A 
365.88 

N/A 



Comparison 
Group 

361.63 
10.48 
370.63 
9.50 



(361.63 - 355.34) ^ 10.48 - 6.29 J- 10.48 - .60 - the pretest gap. 
(370.63 - 365.88) ^ 9.50 - 4.75 i 9.50 - .50 - the posttest gap. 
.60 - .50 - .10 - the gap reduction. 

370.63 - 361.63 - 9.00 ■ Che comparison group's unstandardized growth 
estimate. 



(10.48)^ + (9.50)^ 



109.83 + 90.25 



.04 " 10.00 " the comparison group'^s pooled standard 
deviation. 

STEP 10: 9.00 7 10.00 « .90 - the comparison group^s standard growth estimate. 
STEP 11: .90 + .10 - 1.00 - the project group^s standardized growth estimate. 
STEP 12: (1.00 .90)100 - lllZ - the Relative Growth Index (RGI). 
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